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Abstract Body 



Background / Context: 

All citizens need a broad range of basic mathematical understanding to make informed 
deeisions in their jobs, households, and eommunities; in addition, eareers require an inereasing level 
of profieieney in mathematies (Glenn Commission, 2000; Kilpatriek, Swafford, & Findell, 2001). 
Yet, sinee the 1970s, a series of assessments of Ameriean students' performance has revealed an 
overall level of mathematieal profieieney well below what is desired and needed (Kilpatriek et ah, 
2001; Mullis et ah, 2000). In recent years, the National Couneil of Teaehers of Mathematies 
(NCTM, 2000; NCTM, 2006) and others have addressed these ehallenges with new standards and 
reforms to improve mathematies education, and some progress has been made at the elementary and 
middle school levels (National Mathematics Advisory Panel, 2008; Fuson, Carroll, & Drueek, 2000). 
Nevertheless, eeonomieally disadvantaged ehildren eontinue to demonstrate significantly lower 
levels of math aehievement in sehool (Bowman, Donovan, & Burns, 2001; Denton & West, 2002). 

These aehievement differenees have roots in early ehildhood. A reeent meta-analysis of data 
from several national studies of sehool achievement found that the strongest predietor of aeademie 
achievement in elementary school is mathematical knowledge at school entry (Dunean et ah, 2007). 
Despite the importanee of mathematieal knowledge at sehool entry, however, ehildren from different 
soeioeeonomie baekgrounds enter elementary sehool at different levels of readiness to learn a 
standards-based mathematies eurrieulum (Clements, Sarama, & DiBiase, 2004; Starkey, 2007). 
Preschool children from low soeioeeonomie status (SES) families possess less extensive 
mathematieal knowledge than their peers from higher SES families (e.g., Jordan, Huttenloeher, & 
Eevine, 1994; Starkey, Klein, & Wakeley, 2004). Moreover, this SES-related gap is present as early 
as three years of age, and widens during the presehool years in the United States (Starkey & Klein, 
2008). The source of the early SES-related gap in mathematieal knowledge is twofold. Children 
from low-ineome families reeeive less support for mathematieal development both at home and in 
presehool. Many low-ineome Ameriean parents provide a narrower range of mathematieal learning 
opportunities than middle-ineome parents provide (Blevins-Knabe & Musin-Miller, 1996). 
Eurthermore, publie presehool programs serving low-ineome families, as eompared to private 
programs serving middle-ineome families, typieally provide fewer learning opportunities and 
supports for early mathematieal development (e.g., Bryant, et ah, 1994; Starkey, 2003). In general, 
publie preschool programs do not use a systematie math eurrieulum, and preschool teachers reeeive 
little training in early ehildhood mathematies (Copley, 2004). In summary, both the home and 
sehool learning environments of low-ineome Ameriean children are less rieh mathematieally than 
learning environments of middle-elass ehildren. 

Due to the SES-related gap in early mathematical knowledge and the need to improve low- 
ineome ehildren’ s readiness for school mathematics, we developed an intervention, Pre-K 
Mathematics, and rigorously evaluated it over several studies. This math intervention has been found 
to be highly effeetive at improving pre-k ehildren’ s mathematieal knowledge relative to a eontrol 
group (Klein, et ah, 2008; also see What Works Clearinghouse website, http://ies.ed.gov/neee/wwe). 
In this presentation, we examine the pre-k mathematies intervention implemented at a customary 
level of seale and at a distanee from the developer. Analyses will examine (1) effeetiveness at seale 
aeross varied eontexts, and (2) elassroom praetiees that mediate, at least in part, effeetive math 
outeomes for young ehildren. 
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Experiment 1 - Main Study 

Purpose / Objective / Research Question / Focus of Study: 

The Main Study of the Pre-K Mathematies Scale-Up Project was a randomized controlled field 
trial that had two principal objectives: (1) to determine whether and how the pre-k mathematics 
intervention continues to be effective when implemented at a customary level of scale and at a 
distance from the developer, and (2) to document the process of implementing the math 
intervention in varied contexts in order to understand under what conditions the intervention is 
effective. 

Setting: 

The Main Study was conducted in varied contexts, which included Head Start and state-funded 
public preschool programs in California, Kentucky, and Indiana. Teachers from 94 classrooms 
at 63 sites (Head Start Centers or public preschools) were included in the experiment. 

Population / Participants / Subjects: 

The sample for the experimental field trial included 744 children in 94 classrooms at 63 
preschool sites. The California sample was comprised of ethnically diverse, low-income urban 
children. The Kentucky/Indiana sample was comprised of predominantly Caucasian, low- 
income rural children. Attrition during the pre-k year was 10.1%. 

Intervention / Program / Practice: Pre-K Mathematics, the curricular intervention used in this 
project, has been given the highest rating of effectiveness by the What Works Clearinghouse. 

Classroom component of intervention. The classroom component provided conceptually 
broad support for the development of children’s informal mathematical knowledge. It consisted 
of a set of small-group math activities with concrete manipulatives, math software, and a math 
learning center in the classroom. The small-group activities have been published as a teacher’s 
manual, Pre-K Mathematics (Klein & Starkey, 2004). The mathematical content of the small- 
group activities were based on developmental research about the nature and extent of early 
mathematical knowledge. Activities with closely related mathematical content were organized 
into units in order to help children make connections among related concepts. Furthermore, units 
were explicitly linked with the National Council of Teachers of Mathematics standards for pre-K 
- grade 2. 

Teachers typically conducted small-group math activities twice a week with groups of 4 - 6 
children for 20-25 minutes. Small-group activities were presented to the pre-K children 
according to a weekly curriculum plan with one new math activity introduced each week during 
the school year. Review weeks were also incorporated into the curriculum plan to accommodate 
children who were absent or had difficulty with a particular activity. 

Home component of intervention. The home component of the intervention provided 
parents with activities to support their children’s mathematical development and complement the 
math support children were receiving at preschool. A Spanish version of the home math 
activities was used with Spanish-speaking families. 

Professional development component. A trainer-of-trainers model that was used to 
implement the math curriculum on the scale of a Head Start program or school district pre-k 
program. Internal facilitators (PD staff from or contracted for the preschool programs) attended 
a Facilitators Institute to learn Pre-K Mathematics and to learn how to provide on-site training 
and support to teachers implementing the math curriculum in their programs. Internal 
facilitators, in turn, helped train the teachers and monitored the fidelity of implementation of the 



2011 SREE Conference Abstract Template 



2 




math curriculum in their classrooms. Teachers had, on average, 14 years of teaching experience, 
and 56% had BA degrees or higher; the remainder had AA degrees or less. 



Research Design: 

The basic research design was a cluster randomization in whieh the 63 sites described above 
were randomly assigned to the intervention and control conditions. The sites within eaeh type of 
program (Head Start and state-funded presehool) within eaeh state (CA and KY/IN) were 
grouped into pairs that were similar with regard to the site (total number of classrooms), type 
(half-day or full-day classrooms), and predominant classroom language (English or Spanish). If 
there were more elassrooms or sites than needed in any grouping, the appropriate numbers were 
selected randomly. 

Data Collection and Analysis: 

A set of instruments was used to assess children’s knowledge outcomes. The CMA (Starkey, et. 
ah, 2004) was used to provide a measure of ehildren’s informal mathematical knowledge across 
a broad range of skills and concepts. It was administered to children individually in the fall and 
spring of the pre-K year. The CMA is eomprised of 9 tasks, with multiple problems per task, 
that assess knowledge in the areas of number, arithmetic, space and geometry, measurement, and 
patterns. The range of difficulty is appropriate for children from ages 3 to 5 years of age 
(presehool to K). The TEMA-3 was used along with the CMA as a standardized measure of 
children’s developing mathematical knowledge. Instruments (e.g., EMCO, teaeher and parent 
questionnaires, Eidelity of Implementation Record) were also administered to collect data on 
potential mediators and moderators of effeets in the classroom and home learning environments. 

Findings / Results: 

Fidelity of implementation. The fidelity of teaehers’ implementation oi Pre-K Mathematics 
small group activities was measured periodically across the sehool year. Overall fidelity could 
possibly range from 0.00 (low fidelity) to 1.00 (high fidelity). Observed fidelity was high both in 
California (state-funded preschool classrooms, .96; Head Start, .94) and in Kentucky/Indiana 
(state-funded preschool classrooms, .94; Head Start, .96). This indicates that, in general, 
teachers implemented small group aetivities as intended by the developer. This implies that the 
amount and quality of the professional development that was provided was suffieient for teachers 
to implement with fidelity. 

Curriculum dosage. Curriculum dosage that intervention children reeeived in the elassroom 
and at home was recorded. Summary statisties and moderator analyses are underway and will be 
presented in the paper. 

Child outcomes. Children’s mathematieal knowledge was assessed at pretest and posttest 
using the CMA and TEMA-3. A 3-level repeated measures ANOVA of CMA composite scores, 
with ehildren nested within classrooms within sites, revealed no signifieant difference in pretest 
scores between conditions. A significant ConditionX Time interaction, F( 1,666)= 17 1.28, 
jCK.OOOl indicated greater change in intervention children’s scores across time. An ANCOVA 
model revealed converging findings. There was a significant difference in adjusted posttest 
seores between the intervention and control conditions on the CMA, /»<. 0001. Intervention 
children’s scores increased from 0.29 at pretest to 0.61 at posttest, an increase of 102%; control 
children’s scores increased from 0.32 at pretest to 0.48 at posttest, an increase of 50%. Thus, 
intervention children’s gains were approximately double those of control children. Mean 
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proportion correct by intervention children (.61) and control children (.48), a mean difference of 
12.77 or .74 standard deviations. An alternative measure is to take the difference in the adjusted 
means, 14.31, divided by the pooled standard deviation, which gives .83 standard deviations. By 
either procedure, the effect sizes are large. 

A parallel set of analyses of children’s raw scores on the TEMA-3 at pretest and poshest 
revealed a significant Condition X Time interaction, F(l,663)=47.66, p<.0001. The effect size 
was moderate, .45 standard deviations. Intervention children’s scores increased from 7.0 at 
pretest to 14.8 at poshest, an increase of 1 11%; control children’s scores increased from 7.4 at 
pretest to 12.5 at poshest, an increase of 69%. 

A parallel set of analyses of children’s raw scores on selected subscales of the Woodcock- 
Johnson-III (Understanding Directions; Letter-Word Identification; Spelling) at pretest and 
poshest found no significant Condition X Time interaction for any of these subscales. 

The intervention effect was consistent across states and program types (Head Start or state- 
funded public preschool). There was no significant effect of state or program type and no 
significant interaction of time with state or program type. This indicates that the intervention was 
robust across varied preschool contexts: Head Start vs. state-funded public preschool programs, 
and California programs serving urban, ethnically diverse families vs. Kentucky/Indiana 
programs serving predominantly rural white families. There was a marginally significant main 
effect for gender, with girls scoring higher on the CMA, in general, than boys F(l,666)=5.45, 
p<.02. Gender did not interact with time or condition. 

Mediation analyses. Mediation analyses revealed that total minutes of math support per day, 
as measured by the EMCO, predicted child gains in mathematical knowledge, over and above 
condition. Specifically, more minutes of math, across all types of math support, were associated 
with greater gains on TEMA, even with condition controlled, F(l,575)=7.10,p<.01. 

Eurthermore, more minutes of one particular type of math support (teacher- scaffolded, small- 
group math instruction), was associated with greater gains on TEMA, even with condition 
controlled, F(l,574)=5.84,p<.02. In contrast, minutes of whole-group math instruction was not 
associated with TEMA gains, F(l,574)=1.92, ns. Mackinnon’s procedure found evidence of 
mediation for total minutes and small-group minutes of support but not for whole-group minutes 
of support. 

Experiment 2 - Sustainability Study 

Purpose / Objective / Research Question / Focus of Study: 

The Sustainability Study of the Pre-K Mathematics Scale-Up Project was conducted during the 
year following the Main Study in the same classrooms. The objective was to determine whether 
the Head Start and state-funded public preschool programs and their teachers were able to sustain 
implementation of the math intervention in an effective manner beyond the initial year in which 
they received professional development. 

Setting: 

This study was conducted in the same Head Start and state-funded public preschool programs in 
California and Kentucky/Indiana as in the Main Study. 

Population / Participants / Subjects: 

The sample for included 326 children in 39 intervention classrooms. The California sample was 
comprised of low-income, ethnically diverse, urban children. The Kentucky/Indiana sample was 
comprised of low-income, predominantly Caucasian, rural children. Attrition during the pre-k 
year was 7.1% 
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Intervention / Program / Practice: 

Intervention teaehers from the Main Study continued implementing the classroom and home 
components of the math intervention with pre-k children during the subsequent year. Fidelity and 
dosage data will be provided. 

Research Design: 

The randomization for the Main Study assigned 48 classrooms and their teachers to the 
intervention condition. Of these, 39 teachers continued teaching in these programs at the pre-k 
level during the subsequent year of the Sustainability Study. The remaining 9 teachers could not 
be included, because they were no longer teaching pre-k children in the participating programs. 

Data Collection and Analysis: 

The measures used in the Main Study were again used the Sustainability Study in order to 
determine the extent to which the implementation and positive child outcomes could be sustained 
by programs and to identify any factors associated with changes in implementation. 

Child outcomes. 

Children’s mathematical knowledge was assessed at pretest and posttest using the CMA and 
TEMA-3. A 3-level repeated measures ANOVA of CMA composite scores, with children nested 
within classrooms within sites, revealed no significant difference in pretest scores between 
conditions. A significant Condition X Time interaction, F(l,527)=68.82,p<.0001 (effect size, .70), 
indicated greater change in intervention children’s scores across time. A parallel set of analyses of 
children’s raw scores on the TEMA-3 at pretest and posttest revealed a significant Condition X 
Time interaction, p<.0001 (effect size, .45). 

Conclusions: 

The professional development model that was employed in the Main Study enabled teachers 
to implement with fidelity and effectively. Main impact analyses indicate that the Pre-K 
Mathematics intervention was highly effective when implemented at a customary level of scale, 
with intervention children experiencing gains in mathematical knowledge about twice as great as 
control children’s gains. The intervention was sufficiently robust that children experienced 
significant mathematical growth across varied contexts, including Head Start and state preschool 
programs serving urban, ethnically diverse families in California and predominantly white, rural 
families in Kentucky and Indiana. Mediation analyses supported our hypothesis that the causal 
influence was transmitted, at least in part, by teacher-scaffolded small group mathematics 
activities. 

The Sustainability Study findings indicated that Head Start and state-funded public preschool 
programs and their teachers were able to sustain implementation of the math intervention in an 
effective manner beyond the initial year in which they received professional development. Gains 
in mathematical knowledge by intervention children were significantly greater than those of 
control children. 

Recommendations for policy makers. Public preschool programs should be given 
encouragement and assistance to implement math curricula that are of proven effectiveness. To 
build capacity in early math, there is a need to provide professional development in early math to 
program trainers as well as preschool teachers. It is also recommended that programs forge a 
closer working relationship with parents, such as having teachers send pre-k math materials 
home to parents. We believe that these changes are necessary to make economically 
disadvantaged children ready for elementary school mathematics. 
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